Doesn't work with everything yet but you can call initialize and finalize.

Input:

# For some reason you need to initialize MPI before using multiple libraries.
# Need to debug this
using MPI
MPI.Initialized() || MPI.Init()
using PETSc
for petsclib in PETSc.LibPETSc.petsclibs
    println(splitpath(petsclib.petsc_library)[end])
    @show PETSc.initialized(petsclib)
    @show PETSc.initialize(petsclib)
    @show PETSc.initialized(petsclib)
    @show PETSc.finalized(petsclib)
    @show PETSc.finalize(petsclib)
    @show PETSc.initialized(petsclib)
    @show PETSc.finalized(petsclib)
    println()
end

Output:

libpetsc_double_real_Int64.dylib
PETSc.initialized(petsclib) = false
PETSc.initialize(petsclib) = nothing
PETSc.initialized(petsclib) = true
PETSc.finalized(petsclib) = false
PETSc.finalize(petsclib) = nothing
PETSc.initialized(petsclib) = false
PETSc.finalized(petsclib) = true

libpetsc_single_real_Int64.dylib
PETSc.initialized(petsclib) = false
PETSc.initialize(petsclib) = nothing
PETSc.initialized(petsclib) = true
PETSc.finalized(petsclib) = false
PETSc.finalize(petsclib) = nothing
PETSc.initialized(petsclib) = false
PETSc.finalized(petsclib) = true

libpetsc_double_complex_Int64.dylib
PETSc.initialized(petsclib) = false
PETSc.initialize(petsclib) = nothing
PETSc.initialized(petsclib) = true
PETSc.finalized(petsclib) = false
PETSc.finalize(petsclib) = nothing
PETSc.initialized(petsclib) = false
PETSc.finalized(petsclib) = true

libpetsc_single_complex_Int64.dylib
PETSc.initialized(petsclib) = false
PETSc.initialize(petsclib) = nothing
PETSc.initialized(petsclib) = true
PETSc.finalized(petsclib) = false
PETSc.finalize(petsclib) = nothing
PETSc.initialized(petsclib) = false
PETSc.finalized(petsclib) = true

I guess I still need to sort out the types (But I think it should be possible with more walking of the syntax tree).

Lazy question, but what's the high-level procedure here? What is now automatic and what still has to be maintained "by hand"? Are the specific things here related to various PETSc components all of the "by hand" flavor, or are those things that have been generated automatically and are then checked in?

I think everything in lib is autogenerated.

As @vchuravy said, the file in lib is generated from the PETSc header files with the code in gen. These produce pretty raw wrappers, that I am then expanding on to make it all a little more friendly to work with.

There is a bunch of stuff we do not currently wrap (look at generator.toml), some of it will need to be sorted out and some of it won't (I think I know / have a plan for how to support most of what will be needed, just waiting to fix it until I am ready to use it / write the test).

There probably some gotchas left in there with respect to system / build dependent types that will need to be sorted out, but right now it does support multiple PETSc builds living side-by-side in Julia.

@psanan shoot me an email if you'd like to VTC to chat about what I am doing, I'd be more than happy to have feedback.

Docs suggestion, re our chat our yesterday. Perhaps somewhere like here one could add language like "PETSc.jl's high-level, easy-to-use interface exposes only some of the functionality of PETSc. However, PETSc.jl includes an automatically-generated wrapper layer which directly exposes more of the PETSc API, which may be of use to existing expert users. See (link to minimal example/test of using the wrappers, directly) for an example".

Thanks for the recommendation. I've put it as part of the PR task list.

I’m a PETSc user (well at least of part of the library). Yet I’m only a beginner when it comes to using clang to port it all.
I did notice in a recent project that it is possible to automatically make julia docs out of doxygen strings (which every routine in petsc has).
It would be fantastic if we can automatically provide help for all of the routines (even when the help is strictly speaking only valid for the c routines)

If you leave out the functions that have ccalls in the current main, there are only a limited number of additional functions remaining that are mainly there to make using petsc more julia-like. Those ones could be gradually expanded to include more functionally, but as long as we call autowrapped routines throughout it will be much easier to maintain this.

So with your generated library, did you try doing include petsc_library.jl? Getting it to load cleanly is the first step. I assume you used the generator.toml that is in this branch (which has a lot of stuff to exclude and such).

I see what you are saying about the auto-generated comments for the wrappers. Wouldn't we still need the docs that are in the Julian interface as it stands right now - or are you thinking those could be completely replaced with this auto-generated stuff?

So with your generated library, did you try doing include petsc_library.jl? Getting it to load cleanly is the first step. I assume you used the generator.toml that is in this branch (which has a lot of stuff to exclude and such).

I didn’t manage to load it just now; I was using a simpler generator.toml shown in one of the comments

I see what you are saying about the auto-generated comments for the wrappers. Wouldn't we still need the docs that are in the Julian interface as it stands right now - or are you thinking those could be completely replaced with this auto-generated stuff?

I think it can be replaced with autogenerated stuff. Multiple dispatch will take care of the rest.

@ViralBShah, @vchuravy

One of the reasons for the wrapper file to be rather complex is that the different versions of the PETSc libraries have different definitions of PetscInt, PetscScalar, etc. which can be 32bit, 64bit etc.
In the main branch, we handle this by interpolating the values of $PetscInt, $PetscScalar into function calls as can be seen here

Yet, this doesn't work for structures, which are instead defined as:

mutable struct DMSTAGSTENCIL{PetscInt}
    loc::DMStagStencilLocation
    i::PetscInt
    j::PetscInt
    k::PetscInt
    c::PetscInt
end

At the time the initial wrapping was done, it wasn't possible to automatically wrap C structures to give something like above. Instead Clang gives something like:

mutable struct DMStagStencil
    loc::DMStagStencilLocation
    i::PetscInt
    j::PetscInt
    k::PetscInt
    c::PetscInt
    DMStagStencil() = new()
end

Question: do you know whether there is a way to automatically add types to the structures using Clang?
If yes, it would greatly simplify the generator.toml file.

EDIT: @albert-de-montserrat pointed out that this is actually done already in wrap.jl. The disadvantage is that it has to be specified manually for every struct as:

(x, s) -> add_struct_type(x, s, ("MatStencil",), ("PetscInt",)),

@ViralBShah it now loads again and some of the tests work. Quite a bit of work remains to automatically wrap the structs (if that is possible at all).

Perhaps @Gnimuc can chime in, if they have the time to look at this.

took a bit of time but I now added a (mostly) automatic wrapper tool that creates Julia-friendly functions of the functions generated by Clang.jl. This includes docstrings and tests, which makes the whole experience of using PETSc from Julia much more pleasant.
This is currently tested for the DMSTAG routines (see dmstag_wrapper.jl).

When I looked at this last, it felt like redoing this on top of master picemeal may be easier than trying to rebase this PR to master (which at least for me felt really complex).

When I looked at this last, it felt like redoing this on top of master picemeal may be easier than trying to rebase this PR to master (which at least for me felt really complex).

Not sure I understand what you mean.

I am trying to port the tests from main into this branch bit by bit, so we can ensure that we have at least the same functionality

Ah - I suppose the merge conflicts will then get resolved when all tests are migrated.

Is this still being worked on or can we close it?

I work on it every few months, but was indeed porting things to a new branch (as this one is a bit large by now)